Black Box Anomaly Detection: Is It Utopian?
نویسندگان
چکیده
Automatic identification of anomalies on network data is a problem of fundamental interest to ISPs to diagnose incipient problems in their networks. ISPs gather diverse data sources from the network for monitoring, diagnostics or provisioning tasks. Finding anomalies in this data is a huge challenge due to the volume of the data collected, the number and diversity of data sources and the diversity of anomalies to be detected. In this paper we introduce a framework for anomaly detection that allows the construction of a black box anomaly detector. This anomaly detector can be used for automatically finding anomalies with minimal human intervention. Our framework also allows us to deal with the different types of data sources collected from the network. We have developed a prototype of this framework, TrafficComber, and we are in the process of evaluating it using the data in the warehouse of a tier-1 ISP.
منابع مشابه
"The Tail Wags the Dog": A Study of Anomaly Detection in Commercial Application Performance
The IT industry needs systems management models that leverage available application information to detect quality of service, scalability and health of service. Ideally this technique would be common for varying application types with different n-tier architectures under normal production conditions of varying load, user session traffic, transaction type, transaction mix, and hosting environmen...
متن کاملLightweight Anomaly Detection System with HMM Resource Modeling
In this paper, a lightweight anomaly detection infrastructure named Anomaly Detection by Resource Monitoring is presented for Information Appliances. We call it Ayaka for short. It provides a monitoring function for detecting anomalies, especially attacks which are a symptom of resource abuse, by using the resource patterns of each process. Ayaka takes a completely application black-box approac...
متن کاملAnomaly Detection with Diagnosis in Diversified Systems using Information Flow Graphs
Design diversity is a well-known method to ensure fault tolerance. Such a method has also been applied successfully in various projects to provide intrusion detection and tolerance. Two types of approaches have been investigated: the comparison of the outputs of the diversified services without any knowledge of the internals of the server (black box approach) or an intrusive observation of the ...
متن کاملGray-Box Anomaly Detection using System Call Monitoring
Many host-based anomaly detection systems monitor a process by observing the system calls it makes, and comparing these calls to a model of normal behavior for the program that the process is executing. In this thesis we explore two novel approaches for constructing the normal behavior model for anomaly detection. We introduce execution graph, which is the first model that both requires no stat...
متن کاملEffective Anomaly Detection with Scarce Training Data
Learning-based anomaly detection has proven to be an effective black-box technique for detecting unknown attacks. However, the effectiveness of this technique crucially depends upon both the quality and the completeness of the training data. Unfortunately, in most cases, the traffic to the system (e.g., a web application or daemon process) protected by an anomaly detector is not uniformly distr...
متن کامل